Software is often developed to run with a wide variety of hardware and system software. The differences between these systems have the potential to create compatibility issues. Testing for these issues is essential to ensure overall system integrity and avoid user complaints.
Human testers may be used to catch compatibility issues. This involves running the software on different system configurations and manually checking the results. Not only is this a tedious, time-consuming, and resource intensive process, but the results may be marred from subjectivity and human error.
Test automation has already proven to reduce the cost and improve the accuracy of graphics testing. For example, automated tools may be used to perform screen captures and image comparisons of the same graphical data rendered on multiple platforms. This allows the tester to quickly determine the correctness of different outputs using a standard method of measurement.
While crude automated audio testing methods exist, these methods do no more than determine the mere existence of audio output. Human testing is still needed to determine if audio output processed correctly. While human ears are relatively well-equipped to catch certain audio defects, such as popping sounds, they are inadequate for other aspects, such as precise tone/pitch differentiation, slight timing differences, or accurately parsing a complex clamor of sounds. Additionally, as previously mentioned, such human testing is tedious, time-consuming and resource-intensive and prone to errors due to subjectivity and human error.
Thus, improved audio test automation techniques are needed in order to not only determine if audio output was generated, but to also evaluate if it was generated correctly. Such techniques would improve test result quality, and reduce human testing resource costs.